Platform, Systems, and Methods for Identifying Property Characteristics and Property Feature Maintenance Through Aerial Imagery Analysis

ABSTRACT

In an illustrative embodiment, methods and systems for automatically categorizing a repair condition of a property characteristic may include obtaining aerial imagery of a geographic region including the property, identifying features of the aerial imagery corresponding to the property characteristic, analyzing the features to determine a property characteristic classification, and analyzing a region of the aerial imagery including the property characteristic to determine a condition classification. The methods and systems may further include determining, using the property characteristic classification and the condition classification, a risk estimate of damage to the property due to one or more disasters and/or a cost estimate of repair or replacement of the property characteristic.

RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationSer. No. 62/398,665, entitled “Platform, Systems, and Methods forIdentifying Property Characteristics and Property Feature MaintenanceThrough Aerial Imagery Analysis,” filed Sep. 23, 2016, herebyincorporated by reference in its entirety.

BACKGROUND

A risk exposure database contains a compilation of as many buildingproperties or characteristics relevant to insurance as possible. Theseproperties can include characteristics like location coordinates,address, slope, and elevation, Other characteristics includeconstruction type, occupancy type, year built and/or year of renovation,building height, soft stories, number of stories, and floor area.Further characteristics can include roof condition, roof shape, roofcovering, roof anchors, roof equipment, cladding, and pounding (distanceto closest building). Some of these characteristics can only be assessedby on-site inspections or by official documentation, but others can bemeasured using visual imagery.

Characteristics addressed in this disclosure include roof shape and roofcondition. In one example, roof shapes can be broken into fivecategories: gambrel roof, gable roof, hipped roof, square roof, and flatroof. Each roof shape has a unique response and damage vulnerability todifferent natural perils like earthquake or wind.

Deep learning involves computational models composed of multipleprocessing layers to learn representations of data with multiple levelsof abstractions. These models can be thought of as a way to automatepredictive analytics. Representation learning is a set of methods thatallows a machine to be fed with raw data and to automatically discoverthe representations needed for detection or classification. Use casesfor deep learning include voice recognition, motion detection,translation, and medical diagnosis. By using deep learning algorithmsand sample datasets, computers can learn to distinguish and classify awide range of characteristics to high levels of accuracy, oftensurpassing the recognition levels of human beings.

One model used for deep learning is the Network in Network modeldescribed in the paper “Network In Network” by M. Lin et al. andpublished in the International Conference on Learning Representations,2014 (arXiv:1409.1556), the contents of which are hereby incorporated byreference in its entirety. Using the Network in Network model, a numberof layers of artificial perception outcomes are generated using microneural networks with complex structures. The artificial perceptionoutcomes are then stacked and averaged to generate a single globalaverage pooling layer for classification.

When applied to visual recognition, deep learning algorithms can breakdown an observation (e.g., an image) in a number of different ways tocharacterize features of the observation. In some examples, deeplearning algorithms can be applied to review images as a set ofintensity values per pixel, or in a more abstract way as a set of edges,regions of particular shape, etc. Some representations may demonstratesuperior performance to others based upon the particular learning task.One of the promises of deep learning is replacing human identificationof features with efficient algorithms for unsupervised orsemi-supervised feature learning and hierarchical feature extraction.

The inventors recognized that deep learning methodology could be appliedto risk exposure database population to analyze aerial imagery andautomatically extract characteristics of individual properties,providing fast and efficient automated classification of building stylesand repair conditions. In combining location-based vulnerabilities withindividual property vulnerabilities identified in part throughclassification of repair conditions of one or more property features,risk of damage due to disaster can be more accurately estimated.

SUMMARY OF ILLUSTRATIVE EMBODIMENTS

The forgoing general description of the illustrative implementations andthe following detailed description thereof are merely exemplary aspectsof the teachings of this disclosure, and are not restrictive.

Systems, methods, and computing system platforms described hereinsupport matching aerial image features of one or more properties tocorresponding property conditions (e.g., maintenance levels of propertyfeatures) through machine learning analysis. In a preferred application,the property condition analysis may be used in estimating damage risk inlight of one or more disaster conditions, such as severe storms. Theanalysis may further aid in estimating costs of repair or replacement ofeach property, in one example, should a disaster cause the estimateddamage. In another example, the analysis may be used to confirm that aproperty has been repaired.

In one aspect, the present disclosure relates to a method forautomatically categorizing a repair condition of a propertycharacteristic, including obtaining an aerial image of a geographicregion including the property; identifying features of the aerial imagecorresponding to the property characteristic; analyzing the features todetermine a property characteristic classification; analyzing a regionof the aerial image including the property characteristic to determine acondition classification; and determining, using the propertycharacteristic classification and the condition classification, a riskestimate of damage to the property due to one or more disasters. Theproperty characteristic classification may be rooftop shape. Analyzingthe features to determine a property characteristic may include applyinga deep learning analysis model to the features. The deep learninganalysis model may be NIN.

In some embodiments, analyzing the region of the aerial image includingthe property characteristic to determine the condition classificationincludes applying a machine learning analysis model to image pixelswithin the region. The machine learning analysis model may include acolor histogram analysis model. The condition classification mayencompass classifications good and bad. Determining the risk estimatemay include applying a disaster risk profile corresponding to a firstdisaster of the at least one disaster and the property characteristic.

In one aspect, the present disclosure relates to a system forautomatically categorizing a repair condition of a propertycharacteristic, the system including processing circuitry and anon-transitory computer-readable medium having instructions storedthereon. In some embodiments, the instructions, when executed by theprocessing circuitry, cause the processing circuitry to obtain an aerialimage of a geographic region including the property; identify featuresof the aerial image corresponding to the property characteristic;analyze the features to determine a property characteristicclassification; analyze a region of the aerial image including theproperty characteristic to determine a condition classification; anddetermine, using the property characteristic classification and thecondition classification, a replacement cost for replacing the propertycharacteristic. The aerial image may be a two-dimensional aerial image.

In some embodiments, the instructions, when executed by the processingcircuitry, cause the processing circuitry to, prior to identifying thefeatures, obtain a shape map image including the property; overlay theaerial image with the shape map image; and determine whether a boundaryof the property as identified by the shape map matches a boundary of theproperty as illustrated in the aerial image. The instructions, whenexecuted by the processing circuitry, may cause the processing circuitryto, upon determining that the shape map does not match the boundary ofthe corresponding property, obtain an alternate aerial image of theproperty. The instructions, when executed by the processing circuitry,may cause the processing circuitry to, prior to identifying thefeatures, assess orthogonality of the aerial imagery. The property maybe a single family home.

In one aspect, the present disclosure relates to a non-transitorycomputer readable medium having instructions stored thereon, where theinstructions, when executed by processing circuitry, cause theprocessing circuitry to receive identification of a property and atleast one property characteristic; obtain an aerial image of ageographic region including the property; and identify respectivefeatures of the aerial image corresponding to each propertycharacteristic of the at least one property characteristic. In someembodiments, the instructions, when executed by the processingcircuitry, cause the processing circuitry to, for each propertycharacteristic, analyze the corresponding features to determine arespective property characteristic classification, and analyze a regionof the aerial image including the respective property characteristic todetermine a respective condition classification. The instructions, whenexecuted by the processing circuitry, may cause the processing circuitryto determine, using the property characteristic classification of eachproperty characteristic and the condition classification of eachproperty characteristic, at least one risk estimate representing risk ofdamage due to disaster.

In some embodiments, the instructions, when executed by the processingcircuitry, cause the processing circuitry to, prior to obtaining theaerial image of the geographic region, determine, based upon the atleast one property characteristic, a preferred image type correspondingto each property characteristic of the at least one propertycharacteristic. The at least one property characteristic may include twoor more property characteristics. The instructions, when executed by theprocessing circuitry, may cause the processing circuitry to, responsiveto determining the preferred image type corresponding to a firstproperty characteristic of the at least one property characteristic is aterrestrial image, obtain a terrestrial image of the geographic regionincluding the property.

In some embodiments, the instructions, when executed by the processingcircuitry, cause the processing circuitry to access one or more knownproperty characteristics. Determining the at least one risk estimate mayinclude determining the at least one risk estimate further based on theone or more known property characteristics. The one or more knownproperty characteristics may include at least one of a property age, aproperty elevation, a property slope, a year built, a year renovated,and a building height.

In some embodiments, receiving identification of the property and the atleast one property characteristic includes receiving, via a network froma remote computing device, the identification of the property. Theinstructions, when executed by the processing circuitry, may cause theprocessing circuitry to provide, via the network to the remote computingdevice in real-time responsive to receiving the at least one propertycharacteristic, the at least one risk estimate.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate one or more embodiments and,together with the description, explain these embodiments. Theaccompanying drawings have not necessarily been drawn to scale. Anyvalues dimensions illustrated in the accompanying graphs and figures arefor illustration purposes only and may or may not represent actual orpreferred values or dimensions. Where applicable, some or all featuresmay not be illustrated to assist in the description of underlyingfeatures. In the drawings:

FIG. 1 is a flow diagram of an example system operational flow formatching aerial image features to corresponding property maintenancelevels;

FIG. 2A illustrates example characteristic classifications andcorresponding risk profile data;

FIG. 2B illustrates examples of condition characteristics andcorresponding example histograms;

FIG. 2C illustrates an example condition profile corresponding to anexample property characteristic type;

FIG. 2D is an example error rate graph comparing training data with testdata in conducting matching aerial image features to correspondingrooftop shapes;

FIG. 3 is a block diagram of an example environment for matching aerialimage topographical features to corresponding property maintenancelevels;

FIG. 4 is a flow chart of an example method for classifying conditioncharacteristics of a property based on aerial image analysis;

FIG. 5A illustrates an example structure shape map of a geographic area;

FIG. 5B illustrates an example two-dimensional aerial image of thegeographic area of FIG. 5A;

FIG. 5C illustrates an example overlay map image of a section of thestructure shape map of FIG. 5A overlaying a corresponding section of theaerial image of FIG. 5B;

FIG. 6 is a block diagram of an example computing system;

FIG. 7 is a block diagram of an example distributing computingenvironment including a cloud computing environment; and

FIGS. 8A through 8C illustrate a series of example user interface screenshots for reviewing property characteristic and maintenance levelinformation.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

The description set forth below in connection with the appended drawingsis intended to be a description of various, illustrative embodiments ofthe disclosed subject matter. Specific features and functionalities aredescribed in connection with each illustrative embodiment; however, itwill be apparent to those skilled in the art that the disclosedembodiments may be practiced without each of those specific features andfunctionalities.

Reference throughout the specification to “one embodiment” or “anembodiment” means that a particular feature, structure, orcharacteristic described in connection with an embodiment is included inat least one embodiment of the subject matter disclosed. Thus, theappearance of the phrases “in one embodiment” or “in an embodiment” invarious places throughout the specification is not necessarily referringto the same embodiment. Further, the particular features, structures orcharacteristics may be combined in any suitable manner in one or moreembodiments. Further, it is intended that embodiments of the disclosedsubject matter cover modifications and variations thereof.

It must be noted that, as used in the specification and the appendedclaims, the singular forms “a,” “an,” and “the” include plural referentsunless the context expressly dictates otherwise. That is, unlessexpressly specified otherwise, as used herein the words “a,” “an,”“the,” and the like carry the meaning of “one or more.” Additionally, itis to be understood that terms such as “left,” “right,” “top,” “bottom,”“front,” “rear,” “side,” “height,” “length,” “width,” “upper,” “lower,”“interior,” “exterior,” “inner,” “outer,” and the like that may be usedherein merely describe points of reference and do not necessarily limitembodiments of the present disclosure to any particular orientation orconfiguration. Furthermore, terms such as “first,” “second,” “third,”etc., merely identify one of a number of portions, components, steps,operations, functions, and/or points of reference as disclosed herein,and likewise do not necessarily limit embodiments of the presentdisclosure to any particular configuration or orientation.

Furthermore, the terms “approximately,” “about,” “proximate,” “minorvariation,” and similar terms generally refer to ranges that include theidentified value within a margin of 20%, 10% or preferably 5% in certainembodiments, and any values therebetween.

All of the functionalities described in connection with one embodimentare intended to be applicable to the additional embodiments describedbelow except where expressly stated or where the feature or function isincompatible with the additional embodiments. For example, where a givenfeature or function is expressly described in connection with oneembodiment but not expressly mentioned in connection with an alternativeembodiment, it should be understood that the inventors intend that thatfeature or function may be deployed, utilized or implemented inconnection with the alternative embodiment unless the feature orfunction is incompatible with the alternative embodiment.

Turning to FIG. 1, a flow diagram of an example system operational flow100 for matching aerial image features to corresponding propertymaintenance levels obtains images (104) at property locations 102 b isillustrated. In some implementations, commencement of at least a portionof the operational flow 100 by the system may be triggered by a propertyclassification request submitted by a user, such as an insurance carrieror real estate investment broker at a dashboard interface screen on acomputing device connected to the system via a network. For example, asillustrated in FIG. 8A, a particular property or, alternatively, aneighborhood of properties, may be selected within a map view userinterface 800. The operational flow 100 may be used to analyze thecondition of features of each property at each property location 102 bbased on aerial imagery 102 c. In some implementations, the operationalflow 100 may aid in estimating costs of repair or replacement of eachproperty characteristic detected in the aerial images, for example. Inanother example, the operational flow 100 may be used to confirm that aproperty location 102 b has been repaired.

The property locations 102 b may represent, in some examples, locationsof properties insured by a particular insurer, locations of propertiesrecently affected by a disaster such as a tornado, hurricane,earthquake, storm surge, or explosion, locations of properties held inan investment portfolio, or locations of properties considered foraddition to an investment portfolio. The property locations 102 b may beidentified, in some examples, by a street address, global positioningsystem coordinates, or other geographic coordinates.

In some implementations, the operational flow 100 obtains images (104)at each property location 102 b. The images can include aerial imagery102 c stored by the system or accessed by the system from a remotenetworked location. The aerial imagery 102 c, for example, may include athree-dimensional or two-dimensional image of a geographic locationincluding at least one of the property locations 102 b. In one example,based upon street address or geographic coordinates, the system mayquery a remote database to obtain recent aerial imagery 102 c includingan image of at least one of the property locations 102 b. The remotedatabase, in some examples, can include private industry databases suchas Google® Earth images by Google, Inc. of Mountain View, Calif. or NTTGeospace Corporation of Japan. In other examples, the remote databasecan include one or more databases of publicly owned organizations suchas the Geospatial Information Authority (GSI) of Japan, the UnitedStates Geological Survey, or the Federal Agency for Cartography andGeodesy of Germany. Aerial imagery can be collected from one or moreremote network locations, in some embodiments, using an Open SourceGeographic Information System (GIS) such as QGIS by the Open SourceGeospatial Foundation (OSGeo). The format for the images of the propertylocations 102 b, in some embodiments, depends upon the format acceptedby the various sources available for aerial imagery 102 c. An exampleaerial image 510 of multiple property locations 102 b is illustrated inFIG. 5B.

In some embodiments, the system may be configured to query multipleremote database systems to obtain at least two aerial images of a givenproperty location 102 b. The aerial images available at the variousdatabases, for example, may differ in resolution and recency of capture.Through collecting two or more images of a particular property, forexample, the system may analyze each image to determine a best qualityimage for use in condition analysis. The condition analysis can includebalancing of multiple factors such as, in some examples, clarity,completeness, and recency.

Further, in some embodiments, the system may be configured to querymultiple remote database systems to obtain both a two-dimensional aerialimage and a three-dimensional aerial image. Different propertycharacteristics may be discerned based upon whether the aerial image iscaptured in two-dimensional or three-dimensional format. Two-dimensionalaerial images, in some examples, can be used to determine locationcoordinates of the property, street name, occupancy type, floor area,existence of skylights, existence of chimneys, roof condition, roofshape, roof covering, roof anchors, roof equipment, and/or pounding.Three-dimensional aerial images, in comparison, can be used to determinelocation coordinates, street name, construction type, occupancy type,year built, building height, soft stories, number of stories, roofcondition, roof shape, roof covering, roof anchors, roof equipment,cladding, and pounding. Where there is an overlap in characteristicsidentifiable using either a two-dimensional image or a three-dimensionalimage, in some embodiments, machine learning analysis of both images canbe combined to provide increase confidence in identification of theindividual characteristics.

In some implementations, the system obtains (104) shape map images 102 aof each of the property locations 102 b. The shape map images 102 a, forexample, include a block shape layout of existing properties in amunicipality, such as urban planning maps used in urban planning anddevelopment. The shape map images 102 a, in another example, can includeblock shape layout maps used in presenting information to a user of acomputer-based navigation system. The shape map images 102 a, in oneexample, can be obtained from Geospatial Information Authority of Japanor Zenrin Co. Ltd. of Japan. An example of an urban planning map 500 isillustrated in FIG. 5A (image obtained from the Geospatial InformationAuthority of Japan, Digital Map 2500 (Spatial Data Framework)).

The shape map images 102 a, in some embodiments, are used to confirmlocation of a particular property. The shape map image 102 a of ageographical area, for example, can be overlaid with a correspondingaerial image 102 c to match properties with images. An example overlaidimage 520 of planning map image 500 of FIG. 5A with aerial image 510 ofFIG. 5B is illustrated in FIG. 5C. In some embodiments, the shape mapimage 102 a may not match in shape or location with the aerial image 102c. As illustrated in FIG. 5C, for example, a shape map outline(illustrated in a thickened line) for a location 522 may not berectangular, while a rooftop shape beneath the outline may berectangular in shape. In some embodiments, analyzing multiple aerialimages 102 c to determine the best quality image may include identifyingone or more aerial images 102 c that substantially align with thecorresponding shape map image 102 a. The overlay of the aerial image 102c with the shape map image 102 a, further, can be used in aiding incropping the aerial image 102 c to focus analysis on a particularproperty location 102 b. For example, turning to FIG. 5C, propertylocation 524 is substantially aligning with the corresponding planningmap image, and the general outline can be used in aiding in cropping ofthe property location 524. In some examples, the property location 524may be identified within the overlaid image 520 when the propertylocation 524 aligns with a property in the overlaid image 520 within apredetermined tolerance in order to account for renovations that mayhave been made to the property 524, such as property additions that mayalter an overall shape of the property.

Returning to FIG. 1, in some embodiments, the shape map images 102 a maybe used to correct or compensate for alignment errors or inconsistenciesof the corresponding two-dimensional aerial images 102 c. For example,an aerial image representing a normal orthophoto angle may not bedirectly centered upon the planning map block. Using the planning mapblock as a guide, for example, the aerial image can be geometricallycorrected to obtain a true orthophoto version of the aerial image.

As illustrated in FIG. 1, in some implementations, the system identifies(106) features of each aerial image 102 c to classify propertycharacteristics. Using machine learning for analysis, for example, thesystem can extract features of the aerial image of a particular propertylocation 102 b. Groupings of extracted features, such as angles,outlines, substantially homogenous fields, etc. can be used to identifyproperty features such as rooftop, swimming pool, chimney, and skylights. Upon extracting image-related features identifiable as aproperty feature (e.g., rooftop), the extracted features may be analyzedto determine one or more property characteristics of that feature (e.g.,type of rooftop). In one example, a swimming pool may be identified asthe property feature of a pool (e.g., various shapes of outlines of aparticular size or greater bordering a substantially homogenous field ofblue), then characterized as a particular shape (e.g., rectangular,round, or kidney bean, etc.), type (e.g., above ground or in-ground),and/or size (e.g., approximate area). The machine learning classifierused in the machine learning analysis, in some embodiments, includes aconvolutional neural network (CNN) to preprocess the aerial image 102 cof the particular property location 102 b and to classify the propertyfeatures as property characteristics 110. Using CNN technology, forexample, the biological process of visually reviewing and identifying anobject or feature of an object is mimicked through a feed-forwardartificial neural network where multilayer artificial perceptionminimizes preprocessing. CNN is a well-established and popularmachine-learning methodology. An example of a CNN processing model isAlexnet. (See Krizhevksy et al. “ImageNet Classification with DeepConvolutional Neural Networks”, Advances in neural informationprocessing systems. 2012, the entire contents of which has beenincorporated herein by reference). In other embodiments, the machinelearning classifier may include a Network in Network (NIN) where anumber of layers of artificial perception outcomes are generated usingmicro neural networks with complex structures. The artificial perceptionoutcomes are then stacked and averaged to generate a single globalaverage pooling layer for classification. NIN has demonstrated superiorperformance outcome to conventional CNN processing. Additionally, NINprocessing is less storage-intensive than CNN processing.

In some implementations, property characteristic profiles 108 are usedin classifying property features. The property characteristic profiles108, for example, may be developed through training the machine learningalgorithms using aerial images (and, in some embodiments, terrestrialimages 102 d) of known property characteristics. Each property feature,for example, may be broken down into multiple classifications. In anillustrative example involving a classification of terrestrial images,cladding can include stone, brick, stucco, shingles, vertical boards,horizontal boards, or metal. The machine learning algorithms, forexample, may generate a percentage confidence in a match between a newaerial image 102 c including a gambrel rooftop and the characteristicprofile 108 of gambrel rooftops.

In an illustrative example, turning to FIG. 2A, extracted angular andoutline features of aerial images 204 may be used to classify a roofshape 202 of the particular property location 102 b. In a first example,a gable roof 202 b includes a central peak upon a generally rectangularoutline, as depicted in corresponding aerial image 204 b. A gambrel roof202 a includes these features of the gable roof 202 b but, asillustrated in aerial image 204 a, the gambrel roof additionallyincludes a “skirt” region surrounding the central gable-shaped portion.The “skirt” region can be identified, in part, based upon its corneredges extending from the four corners of the gable-shaped portion. Ahipped roof 202 c, as shown in aerial image 204 c, includes a centralpeak and two opposing triangular ends extending from either end of thecentral peak. The central peak forms a top edge of two trapezoidalsections. A square roof 202 d, in contrast, has no central peak.Instead, as illustrated in aerial image 204 d, the square roof includesfour triangular portions meeting at a central point. Finally, a flatroof 202 e, as shown in aerial image 204 e, is represented by asubstantially flat surface outlined by a rectangle.

Turning to FIG. 2D, an example error rate graph 240 compares trainingdata 242 with accuracy of matching test data 244 in matching aerialimage rooftop characteristics to corresponding rooftop type (e.g.,shape). A test was conducted using NIN deep learning algorithms toclassify rooftop shape in two-dimensional aerial images including a mixof gambrel, gable, hipped, square, and flat roof shapes of assortedcolors. Further tests involved differing image sizes and mini-batchsizes. In particular, images were scaled using a formula ofy=w*(x−100)+z, where x represents a property size (e.g., building with asmall border surrounding the area of the rooftop), y represents pixelsize, w ranged from 0.33 to 0.40 and z ranged from 90 to 110. The errorrates ranged, among initial test cases involving mini batches from 5 to20, from about 4.3% to about 10%. As shown by the error rate graph 240,in a test situation involving 2,250 sample images in a training datasetand 250 test images in a test dataset, the minimum error rate indetermining rooftop condition was 6%, corresponding to an accuracy rateof 94%.

To improve accuracy rate, in some implementations, the image can becropped and/or resized prior to analyzing. For example, the image may becropped to include the property of interest, or the property of interestplus a portion of its surroundings (e.g., the lot containing theproperty and/or a portion of the proximate neighborhood in which theproperty resides). Resizing to a standard image size, in anotherexample, can contribute to accuracy improvement.

Further to roof shape, in some embodiments, feature analysis can be usedto discern additional roof features such as, in some examples, roofcovering, roof anchors, roof equipment, skylights, widow's walks,turrets, towers, dormers, and/or chimneys. Furthermore, upon identifyingthe outline of the roof, a footprint of the property location 102 b(e.g., size of the roof) can be calculated based upon a scale of theaerial image 102 c.

Returning to FIG. 1, rather than exclusively analyzing the particularproperty locations 102 b individually, in some implementations, theparticular property locations 102 b are each classified based in partupon nearby structures. For example, based upon property location 102 bdimensions, which may include lot dimensions, machine learning can beused to identify additional structures at the property location 102 bsuch as, in some examples, a deck, swimming pool, shed, gazebo, detachedgarage, tennis court, fence, retaining wall, dock, playground equipment,or greenhouse. Further, the particular property locations 102 b may beclassified in part based upon distance from neighboring structures(pounding).

In some implementations, terrestrial images 102 d may be used toclassify features difficult to recognize using aerial imagery 102 c. Theterrestrial images 102 d, in some examples, can include street viewimages obtained from a street view service or real estate listing. Insome examples, street view images including a street-facing view of theproperty location 102 b may be obtained from Google® Street View byGoogle, Inc., Bing® Maps Streetside by Microsoft Corp. of Redmond,Wash., or Mapillary by Mapillary AB of Sweden. Using a terrestrialimage, for example, the system can identify features such asconstruction type, cladding, building height, number of soft stories,number of floors, location coordinates, street name, slope, elevation,year built, and/or occupancy type. Where the characteristicsidentifiable via the terrestrial imagery 102 d overlap withcharacteristics identifiable via two-dimensional or three-dimensionalaerial imagery 102 c, analysis of terrestrial imagery 102 d can becombined with analysis of aerial imagery, in some embodiments, toincrease confidence in identification of the particularcharacteristic(s). For example, house siding features may be more easilydetected in terrestrial imagery 102 d and/or three-dimensional aerialimagery 120 c than in two-dimensional aerial imagery 102 c.

In some implementations, a condition of each property feature may beclassified (112) as a corresponding condition characteristic 116. Newproperties are in good condition, but property feature conditions candeteriorate over time due to normal wear-and-tear on the property.Further, property features can suffer damage due to external forces suchas storms and natural disasters. Eventually, conditions of housingfeatures can deteriorate to the point where repair and/or replacementmay be necessary. As with property characteristics described above atblock 106, machine learning algorithms can be used to classify a presentcondition of individual detected property features. Using machinelearning for analysis, for example, the system can extract pixelintensity distributions of previously identified property features ofthe aerial image of the particular property location 102 b. In someexamples, newly constructed property features generally have sharpcontrast and well-defined features in machine learning image analysis.Conversely, weathered or damaged property features can have softenededges, blurred contrasts, and asymmetrical patches of wear. The machinelearning classifier used in the machine learning condition analysis, insome embodiments, includes a machine learning analysis to process theaerial image 102 c of the particular property location 102 b and toclassify the condition of previously identified property characteristics110 as condition characteristics 116. The machine learning analysis, insome examples, can include two-dimensional color histogram analysis orthree-dimensional color histogram analysis. In other embodiments, themachine learning analysis may be performed using pattern recognitionalgorithms (e.g., determining missing fence posts or missing/misalignedrooftop shingles). In other embodiments, the machine learning classifierincludes deep learning analysis such as CNN or NIN.

In an illustrative example, turning to FIG. 2B, examples of conditioncharacteristics of aerial images of hipped rooftops and correspondingexample histograms generated through machine learning analysis of pixelintensity are presented. For example, a set of good condition aerialimages 210 a through 214 a, are presented alongside corresponding goodcondition histograms 210 b through 214 b. The good condition histograms210 b through 214 b, in an illustrative embodiment, illustrateprobability metrics of grayscale color distribution, where a respectivemaximum value 210 c through 214 c demonstrates the peak probability ofgrayscale color distribution of a good condition roof. The values, forexample, can range from 0 (total black) to 255 (total white).

Conversely, a set of poor condition aerial images 216 a through 220 a,are presented alongside corresponding poor condition histograms 216 bthrough 220 b. The poor condition histograms 216 b through 220 b, in anillustrative embodiment, illustrate probability metrics of grayscalecolor distribution, where a respective maximum value 216 c through 220 cdemonstrates the peak probability of grayscale color distribution of agood condition roof. The maximum values 216 c through 220 ccorresponding to the poor condition rooftops 216 a through 220 a aremarkedly lower than corresponding maximum values 210 c through 214 c ofgood condition rooftops 210 a through 214 a. In addition, a width of thedistributions of the poor condition histograms 216 b through 220 b maybe markedly wider than a width of the distributions of the goodcondition histograms 210 b through 214 b. As can be seen of the aerialimages of the poor condition rooftops 216 a through 220 a, the colordistribution is patchy and bleached out, while the edges of the rooftopshave lost their crisp lines. Conversely, looking to the aerial images ofthe good condition rooftops 210 a through 214 a, the rooftops are moresubstantially uniform in color with crisp lines on the edges.

Turning to FIG. 2C, a condition profile 230 for a rooftop condition isillustrated. In some implementations, pixel value distributionhistograms such as the histograms 210 b through 220 b, in someimplementations, can be combined to generate an example conditionprofile 230 corresponding to rooftop condition. The condition profile230, for example, may be included as one of the condition profiles 114of FIG. 1. Example reference bars 232 a, 232 b on the condition profile230 illustrate values corresponding to a non-deteriorated hipped roof214 a and the deteriorated hipped roof 216 a, respectively. In someexamples, the closer the value of a reference bar 232 comes to 1 on theY-axis, the better the condition of the roof, while values close to 0represent poor condition rooftops. In one example, the reference bars232 of the example condition profile 230 correspond to a test conditionssample data set including 2,500 images). The reference bars 232 can becombined, in some embodiments, in a logistic regression to generate aroof condition profile graph 236. The accuracy rate of the imagehistogram is 80% based on some validation.

Although illustrated as a single condition profile 230 corresponding todifferent types of rooftops (e.g., at least the hipped rooftop 216 a andthe gable rooftop 234), in other embodiments, individual conditionprofiles 114 can be generated for each property characteristic (e.g.,profiles specific to gabled, gambrel, flat, hipped, square, etc.). Infurther refinements, in certain embodiments, individual conditionprofiles 114 can be generated for combinations of propertycharacteristics (e.g., a gabled, shingled roof, a gabled, clay tiledroof, a gabled, metal roof, etc.) to increase accuracy rate based uponpixel densities corresponding to the combined characteristics. Manycombinations of characteristics of property features may be used singlyor in combination to generate a condition profile 114 designed toaccurately identify the condition of the corresponding property feature.

Returning to FIG. 1, in some implementations, the propertycharacteristics 110 and/or condition characteristics 116 may be analyzedto calculate (120) a risk estimate 122 corresponding to a risk of damagebased upon one or more types of disasters. The types of disasters caninclude, in some examples, earthquake, hurricane, tornado, storm surge,straight line winds, or explosion. The types and estimated severity ofdisasters, in some embodiments, may depend upon the particular propertylocation 102 b. For example, properties near active fault lines may bemore susceptible to earthquakes, while coastal properties may be moresusceptible to storm surge damage. The risk estimate 122, in someexamples, can include an actuarial percentage indicating a likelihood ofdamage based upon the particular property characteristics. Differentproperty characteristics, for example, may be more susceptible thanothers to certain types of damage. For example, flatter rooftops may bemore susceptible to damage from excessive snow build-up, while peakedrooftops may be more vulnerable to strong winds.

In some embodiments, in calculating damage risk, one or more disasterrisk profiles 118 can be applied based upon property characteristics.Vulnerability to damage due to disaster can vary, in some examples, bylocation, altitude, slope, rooftop shape, rooftop condition, claddingtype, cladding condition, and/or pounding. In a particular example, asshown in FIG. 2A, a graphical risk profile 206 ranks types of rooftopsby their vulnerability to strong winds. Vulnerability to wind damageincreases from lowest vulnerability (flat rooftop 202 e) to square roof202 d, to hipped roof 202 c, to gable roof 202 b, with the greatest winddamage vulnerability in gambrel roofs 202 a. Vulnerability may bemathematically modeled based upon statistical information built frompast damage results due to varying types and/or severity of disaster. Incombining location-based vulnerabilities with individual propertycharacteristic-based vulnerabilities, risk of damage due to disaster canbe more accurately estimated.

Referring back to FIG. 1, the risk estimates 122, in some embodiments,can be used in insurance assessments. For example, the risk estimate 122may be used in determining whether a property is adequately insuredagainst potential damage. Further, when tracking an impending naturaldisaster, an insurance company may use the risk estimates 122 toestimate claims based upon the forecast path and severity of the storm.Additionally, the risk estimates 122 may be applied by investors whenselecting investment properties for an investment portfolio.

In some implementations, repair or replacement costs can be calculated(126) to determine a cost estimate 128 to place one or more propertycharacteristics in good or “like new” condition. The cost estimates 128,in some embodiments, can be based in part on one or more replacementcost profiles 120. Replacement costs, in some examples, may vary basedupon type of materials (e.g., rooftop materials, cladding materials,etc.), size of the job (e.g., estimated area of the roof, estimatedlength of fence, etc.), and/or additional property characteristics(e.g., contributing to the complexity of the work). The replacementprofiles 120, in further examples, may be based in part on location(e.g., varying costs of material and labor regionally). In someimplementations, the cost estimates 128 for repair or replacement of theproperty characteristics may be output to a dashboard interface screenat a remote computing device of a requester, such as an insurancecarrier or real estate investment broker.

In some embodiments, cost estimates 128 can be used in automaticallyanalyzing insurance claims. The cost estimate 128, further, may becombined with the risk estimate 122 in analyzing adequacy of insurancecoverage for a particular property location 102 b. Further, the costestimates 128 may be used by investors in determining cost ofmaintaining/upgrading a particular investment property.

Although the operational flow is illustrated as a series ofcomputational stages, in other implementations, more or fewercomputational stages may be included. For example, aerial imagery 102 cmay be analyzed after an insurance claim has been paid to verify thatthe insured property has been repaired and/or to upgrade the estimatedvalue of the property (e.g., in the circumstance where the ownerupgraded the property rather than applying a straight repair/replacementof the damaged property characteristics.

Additionally, in other implementations, certain computational stages maybe performed in a different order. For example, cost estimates 128 maybe calculated prior to risk estimates 122. Other modifications of theoperational flow 100 are possible.

FIG. 3 is a block diagram of an example environment 300 for matchingaerial image topographical features to corresponding propertymaintenance levels. The example environment 300, for example, can beused to implement the operational flow 100 described in relation toFIG. 1. The example environment 300 includes a property characteristicclassification and condition analysis system 302 in communication withclient computing system(s) 306, a number of cost data and/or risk datasource computing system(s) 308, and a number of shape map, aerialimagery, and/or terrestrial imagery source computing system(s) 304. Thecommunication connection may be wired or wireless, over a variety ofnetwork types. In a general overview, the system 302 obtains propertyidentifier(s) 340 from client computing system(s) 306, accesses imagesof a property identified by the property identifier 340 from the imagerysource computing system(s) 304, and analyzes the images to determineproperty characteristic(s) 342 and condition characteristic(s) 344.Using cost and/or risk data obtained from cost and risk data sourcecomputing system(s) 308, the system 302 may determine replacement costestimate data 350 and/or disaster risk estimate data 352 based upon theproperty characteristics and their condition.

In some implementations, a user of a particular client computing system306 submits a request to the system 302 through a graphical userinterface supplied by a graphical user interface engine 334. Therequest, for example, may include at least one property identifier 340as well as one or more property characteristics 342 and/oridentification of at least one of a replacement or repair cost estimate,a disaster risk estimate, and a confirmation of repair assessment. Theproperty identifier 340, in some examples, may include locationinformation (e.g., address, geolocation coordinates, lot boundaries,etc.). The location information, for example, may conform with propertylocation information 102 b described in relation to FIG. 1. In otherembodiments, the property identifier 340 may be a unique propertyidentifier (e.g., number or string) used to look up property locationinformation in a data store 310. In another embodiment, rather thansubmitting a property identifier, the user may submit a unique useridentifier used to look up one or more properties associated with theuser (e.g., properties insured by an insurance client, properties heldby an investment client, etc.).

In one example, turning to FIG. 8A, a user may identify locations of oneor more properties by selecting the properties within a graphical mapinterface 800. A number of properties are identified within the map viewuser interface 800 as indicated with an associated pin, such as pin 804.The pins, in some examples, may indicate properties insured by aparticular insurance carrier, a type of property (e.g., commercial,industrial, residential, etc.), and/or properties of a value range(e.g., within a range, above a threshold, etc.). The pins, for example,may be displayed based upon filtering options supplied by the user orotherwise associated with a user profile (e.g., insurance carrieridentification). In a particular example, a user may select threeproperties associated with three dialogue bubbles 802. As illustrated,each dialogue bubble identifies a replacement value associated with theproperty, and invites the user to review details regarding the selectedproperty via a details control.

In some implementations, the user enters property characteristicsassociated with each selected property. In other implementations, thecharacteristics may be accessed upon identification of the property(e.g., selection of a pin identifying a property, as illustrated in FIG.8A). For example, the property characteristics 342 may be obtained fromthe data store 310, as illustrated in FIG. 3. Turning to FIG. 8B, anexample property characteristics user interface 820 presents buildingarea characteristics 822 (e.g., building area 822 a, stories 822 b, andtotal building area 824 c) and structure characteristics 824 (e.g.,structure material 824 a, construction class 824 b). As describedwithin, different or additional characteristics may vary perimplementation. In some embodiments, the user may be provided theability via the user interface 820 to edit the building areacharacteristics 822 and/or the structure characteristics 824, forexample via an edit control 828 a or 828 b. For example, in thecircumstance of recent development or renovation (e.g., construction ofan addition), the user may modify previously stored characteristicsvalues. The property characteristics user interface 820, for example,may be presented to a user upon selection of a details control of one ofthe dialogue bubbles 802 illustrated in the user interface of FIG. 8A.

The user interface 820 further presents, in some implementations,premium characteristics 826 such as a replacement value. The premiuminformation, for example, may relate to an insurance policy purchasedfor the property or parameters for insuring the property (e.g., basedupon upgraded or new characteristics). The user, in some embodiments,may be provided to update the premium information via an edit control828 c.

In some implementations, upon identification of property locationinformation, the system 302 accesses imagery of the property from theimagery source computing system(s) 304 using an image acquisition engine336. The images can include, in some examples, a shape map (e.g., suchas the shape maps 102 a described in relation to FIG. 1),two-dimensional aerial imagery (e.g., such as the aerial imagery 102 cdescribed in relation to FIG. 1), three-dimensional aerial imagery(e.g., such as the aerial imagery 102 c described in relation to FIG.1), and/or terrestrial imagery (e.g., such as the terrestrial imagery102 d described in relation to FIG. 1). The images obtained by the imageacquisition engine 336, in some embodiments, depend upon the propertycharacteristic(s) identified by the user. For example, certain types ofimages are preferred for analyzing certain types of propertycharacteristics. Housing siding, for example, is more easily perceivedfrom a terrestrial image or three-dimensional aerial image than from atwo-dimensional aerial image.

In some implementations, an image quality analysis and preparationengine 326 analyzes the acquired image(s) to confirm the acquiredimage(s) contains an adequately clear and detailed image of the propertyidentified by the property identifier 340. In some embodiments, theimage quality analysis and preparation engine 326 crops and/or resizesthe acquired image(s). For example, the image quality analysis andpreparation engine 326 may extract a portion of an acquired image basedupon the property (e.g., building size). In some implementations, theimage quality analysis and preparation engine 326 resizes each acquiredimage to a standard size. The standard size, in a particular example,may be 256 by 256 pixels per image. Resizing to a standard size, forexample, may increase accuracy in later classification analysis. Theimage quality analysis and preparation engine 326, in some embodiments,may be configured to apply corrections to the acquired image(s). Forexample, the image quality analysis and preparation engine 326 may beconfigured to adjust an aerial image from a normal orthophoto angle, toa true orthophoto version as described above.

In some implementations, the image quality analysis and preparationengine 326 uses shape outlines as part of the analysis. For example, acomposite image generation engine 318 may overlay an aerial image with ashape map image, as described in relation to FIG. 5C. Using theresultant composite image, the image quality analysis and preparationengine 326 may confirm the aerial image conforms with an urban planningmap or other property placement guide (e.g., navigational map data,etc.). Further, the image quality analysis and preparation engine 326may use the overlaid shape map as a guide in determining whether anaerial image requires correction to a true orthophoto angle and/or howclosely to crop the image to capture the property (and, optionally, itsimmediate surroundings such as its lot or a portion of the proximateneighborhood).

In some implementations, if the image quality analysis and preparationengine 326 determines that the acquired image is insufficient, the imagequality analysis and preparation engine 326 may request a replacementimage from the image acquisition engine 336. For example, the imageacquisition engine 336 may obtain images based upon a variety of factorsincluding, in some examples, recency of capture, resolution, cost,and/or applicability to a particular property characteristic analysis.Upon determination by the image quality analysis and preparation engine326 that the first obtained image is insufficient, for example, theimage acquisition engine 336 may determine a next best source forobtaining an image of the property.

In some implementations, once an image has been approved (and,optionally, prepared and/or corrected) by the image quality analysis andpreparation engine 326, a feature identification engine 320 extractsfeatures from the property image(s) related to the identified propertycharacteristics 342. In the circumstance of a fence, for example, thefeature identification engine 320 may identify a perimeter enclosure orpartial perimeter enclosure abutting and extending from the property asindicative of a fence.

In some implementations, upon identification of features by the featureidentification engine 320, a property characteristic classificationengine 322 classifies the property characteristic. For example, thefeatures may be classified using one or more machine learning techniquesas described in relation to computational stage 106 (classify featuresof each image) of FIG. 1. The property characteristic classificationengine 322, for example, may generate property characteristics 342representing a grouping, type, or other characterization of a particularproperty feature. Continuing with the particular example of a fence, theextracted features may be analyzed and classified in relation to heightand/or material. Upon classifying the property characteristics, theproperty characteristic classifications may be stored in the data store310 as property characteristics 342 (e.g., similar to propertycharacteristics 110 of FIG. 1).

In some implementations, the property characteristic classificationengine 322 uses characteristic profile(s) 360 in classifying propertycharacteristics. The property characteristic classification engine 322,for example, may obtain, for each property characteristic beinganalyzed, a particular characteristic profile 360 from a profile datastore 314. The characteristic profiles, for example, may be similar tothe characteristic profiles 108 described in relation to FIG. 1.

In some implementations, the property characteristic profiles 360 aregenerated by a characteristic classification learning engine 328. Thecharacteristic classification learning engine 328, for example, maygenerate characteristic profiles 360 through a sample data set andlearning process as described, generally, in relation to FIG. 2A. In aparticular example, the characteristic classification learning engine328 may analyze JPG (lossy compression digital image standard developedby the Joint Photographic Experts Group) to recognize varying rooftopshapes.

While certain property characteristic classifications are done byanalysis, in some implementations, one or more feature classificationsmay be obtained from previously stored property characteristics 342. Forexample, based upon tax records, real estate records, etc. some basicmaterials and structure information regarding a property may be readilyavailable through local and/or remote database system(s). In otherexamples, the previously stored property characteristics 342 may also beobtained from previous condition assessments performed for a propertylocation by the system 302.

In some implementations, once the property characteristic classificationengine 322 has classified the property characteristic(s), acharacteristic condition classification engine 324 classifies thecondition of each property characteristic. For example, the propertycharacteristics may be classified as described in relation tocomputational stage 112 (classify condition of each feature) of FIG. 1.The characteristic condition classification engine 324, for example, maygenerate condition characteristics 344 representing a rating,qualification, or ranking of the condition of the particularcharacteristic. In some examples, the characteristic conditionclassification engine 324 may qualify a characteristic as a) “good” or“bad”, b) “like new”, “acceptable”, “needs repair”, or “needsreplacement”, c) rated on a numeric scale such as a scale from 1 to 5where 5 may correspond to “like new” and 1 may correspond to “needsreplacement.” In other implementations, particular problems may beidentified by the characteristic condition classification engine 324such as, in some examples, a fence feature is partially fallen, or aroof feature shows signs of structural damage.

In some implementations, the characteristic condition classificationengine 324 uses condition profile(s) 362 in classifying propertycharacteristic conditions. The characteristic condition classificationengine 324, for example, may obtain, for each property characteristicbeing analyzed, a particular condition profile 362 from the profile datastore 314. The condition profiles 362, for example, may be similar tothe condition profiles 114 described in relation to FIG. 1.

In some implementations, the condition profiles 362 are generated by acondition classification learning engine 330. The conditionclassification learning engine 330, for example, may generate conditionprofiles 362 through a sample data set and learning process asdescribed, generally, in relation to FIG. 2B. In a particular example,color histogram analysis may be used to identify peak pixel values ofthe images in gray scale. Using the group of calculated peak pixelvalues, the learning engine can classify property condition into certaincategories.

In some implementations, based upon the output of the propertycharacteristic classification engine 322 and/or the characteristiccondition classification engine 324, a risk calculation engine 316 mayestimate risk of damage based upon one or more types of disaster. Forexample, the disaster risk estimate data 352 may be calculated asdescribed in relation to computational stage 120 (calculate damage risk)of FIG. 1. The disaster risk estimate data 350, in one example, caninclude a percentage likelihood of damage due to each particular type ofrisk based upon the particular property characteristics and/or propertyconditions, as described in relation to the risk estimates 122 ofFIG. 1. Further, in some embodiments, the disaster risk estimate data350 may include a set of risk estimates based upon, in some examples, anumber of severities of disaster, a particular path of disaster, and/orhazard intensity of disaster. The disaster severities, in some examples,may correspond to the Richter scale of seismic activity magnitude or theSaffir-Simpson Hurricane Wind Scale category. The path of the disaster,in some examples, can include geographical direction such asnortheasterly, etc. or angle of impact with respect to the propertycharacteristic(s). The hazard intensity of disaster, in some examples,can relate to wind speed or rainfall amount. Repair costs, for example,may be more accurately estimated when the roof condition is known. Dueto poor condition of a rooftop in one particular example, the roof maybe more vulnerable to damage requiring replacement. In anotherparticular example, due to a poor condition rooftop, the likelihood forinterior property damage (e.g., water damage due to breach of the roof)may increase at a set level of severity of disaster and above.

In some embodiments, the risk calculation engine 316 accesses one ormore risk profiles 364 from the data store 314 based upon propertycharacteristics and/or condition characteristics. The risk profiles, forexample, may be similar to the disaster risk profiles 118 described inrelation to FIG. 1. The risk profiles 364, for example, may be obtainedfrom one or more risk data sources 306 by a cost and risk dataacquisition engine 338. In other embodiments, the system 302 maygenerate risk profiles 364 (e.g., using risk data obtained from one ormore cost and risk data sources 306 and a risk data analysis engine (notillustrated)).

The risk estimate data 352, in some embodiments, is shared with one ormore clients 306. For example, an insurance carrier client 306 may userisk estimate data in making insurance assessments. In another example,a real estate investment broker or firm client 306 may apply riskestimate data 352 when selecting investment properties for an investmentportfolio.

Turning to FIG. 8C, an example disaster risk user interface 840 presentsrisk assessment information based upon risk profiles 364 related to theselected property (e.g., one of the properties identified via thedialogue bubbles 802 c of FIG. 8A). The user interface 840, for example,may be accessed by scrolling downward from the user interface 820. Theuser interface 840 illustrates a number of disaster risk factors 842 andassociated risk estimate data 844 for the selected property. Asillustrated, the disaster risk factors 842 include a fire factor 842 a,a wind factor 842 b, a flood factor 842 c, an earthquake factor 842 d, atsunami factor 842 e, and a landslide factor 842 f. In some embodiments,only a subset of the risk estimate data is determined via riskassessment techniques described herein. For example, flood data 844 cmay be generated based upon governmental flood map evaluation ratherthan via aerial imaging assessment, while fire risk 844 a may bedetermined at least in part based upon proximity to neighboringproperties as derived via imaging analysis and/or constructionproperties derived via image analysis. Further, wind risk data 842 b maybe derived via rooftop analysis as described in greater detail herein.

In some implementations, based upon the output of the propertycharacteristic classification engine 322 and/or the characteristiccondition classification engine 324, a cost calculation engine 336calculates repair or replacement costs for improving problems identifiedby the characteristic condition classification engine 324 and generatesreplacement cost estimate data 350. For example, the replacement costestimate data 352 may be calculated as described in relation tocomputational stage 126 (calculate repair or replacement costs) ofFIG. 1. The replacement cost estimate data 350, for example, may besimilar to the cost estimates 128 described in relation to FIG. 1.

In some implementations, the cost calculation engine 336 basedcalculations in part on one or more cost profiles 366 (e.g., similar tothe cost profiles 120 of FIG. 1). In some embodiments, the costcalculation engine 336 accesses one or more cost profiles 366 from thedata store 314 based upon property characteristics and/or conditioncharacteristics. The cost profiles 366, in some embodiments, areobtained from one or more cost data sources 306 by the cost and riskdata acquisition engine 338. In other embodiments, the system 302 maygenerate cost profiles 366 (e.g., by a cost data analysis engine (notillustrated) using cost data obtained from one or more cost and riskdata sources 306).

The cost estimate data 350, in some embodiments, is shared with one ormore clients 306. For example, an insurance carrier client 306 may usecost estimate data in determining adequacy of insurance coverage for aparticular property. In another example, a real estate investment brokeror real estate investment firm client 306 may apply cost estimate data350 in determining cost of maintaining/upgrading a particular investmentproperty.

In some implementations, the cost estimate data 350 and/or risk estimatedata 352 is provided to a requesting client 306, responsive to receivingidentification of one or more properties, in real-time or nearreal-time. For example, a client 306 accessing a dashboard interface,may format a request for a risk estimate or cost estimate related to anidentified property within a dashboard interface and submit the requestvia a network to the system 302. Responsive to the request, the system302 may conduct the analysis described generally above and respond, inreal time or near-real time to the client 306 with a risk analysis orcost analysis. For example, risk analysis information is presented inuser interface 840 of FIG. 8C.

In some implementations, the system may identify, through updatedmeteorological data, one or more properties within a client real estateportfolio affected by a disaster. The system, further to identificationof the affected properties, may proactively prepare a repair estimate toprovide to the client corresponding to the real estate portfolio shortlyafter the disaster took effect. In one example, the Japan MeteorologicalAgency may update observation data via a web site interface within anhour or so of observation. In this example, the system may providerepair cost estimates to clients between one and two hours after adisaster has taken place.

Rather than sharing cost estimate data 350 and/or risk estimate data 352directly with the clients 306, in other embodiments, the system 302 mayinclude a report generation engine (not illustrated) that preparesreports regarding condition, damage, and risk assessments of one or moreproperties. Further, in some embodiments, the system 302 may compare acurrent condition characteristic 344 to a historic conditioncharacteristic 344 to confirm whether a property owner made repairs to aproperty (e.g., based upon payment of an insurance claim). Othermodifications of the system 302 are possible.

FIG. 4 is a flow chart of an example method 400 for classifyingcondition characteristics of a property based on aerial image analysis.The flow chart, for example, may represent computational stages 104through 112 as described in relation to FIG. 1. The method 400 may beperformed, for example, by the system 302 of FIG. 3.

In some implementations, the method 400 begins with accessing atwo-dimensional shape map including the shape of a property (402). Theshape map, for example, may be accessed by the image acquisition engine336 from a shape map imagery source 304, as described in relation toFIG. 3. The two-dimensional shape map, for example, may take a form ofany of the shape maps 102 a described in relation to FIG. 1. In aparticular example, the shape map may be in a form similar to the shapemap 500 described in relation to FIG. 5A.

In some implementations, two-dimensional and/or three-dimensional aerialimagery including an image of the property is accessed (404). Thetwo-dimensional and/or three-dimensional aerial imagery, for example,may be accessed by the image acquisition engine 336 from an aerialimagery source 304, as described in relation to FIG. 3. Thetwo-dimensional and/or three-dimensional aerial imagery, for example,may take a form of any of the aerial imagery 102 c described in relationto FIG. 1. In a particular example, the two-dimensional aerial imagerymay be in a form similar to the aerial image 510 described in relationto FIG. 5B.

In some implementations, the aerial imagery is overlaid with the shapemap (406). The composite image generation engine 318 of FIG. 3, forexample, may overlay the aerial imagery with the shape map to generate acomposite image. The composite image, for example, may be in a formsimilar to the composite image 520 described in relation to FIG. 5C.

If a shape match is not identified (408), in some implementations, themethod 400 accesses alternate two-dimensional or three-dimensionalaerial imagery (410). In some embodiments, the image quality analysisand preparation engine 326 of FIG. 3 determines whether the compositeimage presents an adequate shape match of the property(s) of interest.Upon determining the image does not match, the image acquisition engine336 of FIG. 3, for example, may obtain an alternative two-dimensional orthree-dimensional aerial image from the aerial imagery source(s) 304. Asdescribed in relation to FIG. 3, for example, For the image acquisitionengine 336 may determine a next best source for obtaining an image ofthe property based upon one or more factors. Conversely, the method 400,in other implementations (not illustrated), could proceed withoutmatching the shape map to the aerial imagery.

If a shape match is identified (408), in some implementations, theformatting of the aerial imagery is prepared (411). In some embodiments,the image is cropped to include the property of interest or the propertyof interest plus a portion of its surroundings (e.g., the lot containingthe property and/or a portion of the proximate neighborhood in which theproperty resides). Cropping may be based, for example, on thecorrelation of the shape map to the property. In illustration, the imagemay be cropped to include the shape map outline of the property plus aborder region. In some embodiments, cropping may include cropping to aset shape (e.g., X by X pixels square, X by Y pixels rectangle, etc.).In some embodiments, in addition to cropping the image, the image may beresized. For example, depending upon the resolution of the aerialimagery, the image resolution may be reduced, for example, to fit withinan X by X pixels square or an X by Y pixels rectangle. Resizing to astandard image size, for example, can contribute to analysis consistencyand improvement in classification accuracy. In further embodiments, thecolor depth and/or color mapping of the aerial imagery may be adjusted.For consistent color histogram analysis of the imagery, for example, thecolor depth and color mapping may be made consistent across analyzedimages. In a particular example, color images may be converted tograyscale for grayscale image analysis. Adjustments to the imagery, forexample, may be effected by the image quality analysis and preparationengine 326, described in relation to FIG. 3.

In some implementations, orthogonality of the aerial imagery is assessed(412). For example, the image quality analysis and preparation engine326 may assess orthogonality of the aerial imagery. The image qualityanalysis and preparation engine 326, in one example, may determine thatorthogonality correction is desired. In another example, if the imagerepresents a normal orthophoto rather than a true orthophoto, the imagequality analysis and preparation engine 326 may notify further moduleswhich may compensate for the angle of capture of the aerial image. Inother implementations (not illustrated), orthogonality may be assessedwithout use of an overlaid shape map.

Returning to FIG. 4, in some implementations, if the aerial imagery isnot orthogonal (414), an orthogonal version of the aerial imagery isgenerated (416). The image quality analysis and preparation engine 326of FIG. 3, for example, may correct a normal orthophoto form to generatea true orthophoto form. The true orthophoto form, for example, maybetter match the shape map overly.

In some implementations, one or more property characteristics of theproperty are classified (418). The property, in some embodiments, isanalyzed based on the portion of the aerial image substantially boundedby the shape map of the property. In other embodiments, the propertyanalysis encompasses a surrounding of the property (e.g., featuresproximate the property upon the property lot, nearby properties, etc.).The property characteristic(s), for example, may be classified by theproperty characteristic classification engine 322, as described inrelation to FIG. 3. In a particular example, a rooftop property featuremay be classified as one of a variety of rooftop shapes, as illustratedin FIG. 2A.

In some implementations, if a property characteristic classification isidentified (420), a condition of each of the property characteristicscan be classified (422). The property characteristic condition(s), forexample, may be classified by the characteristic conditionclassification engine 324, as described in relation to FIG. 3. In aparticular example, a hipped classification of the rooftop feature maybe classified as one of a good condition and a poor condition, asillustrated in FIG. 2B. In some implementations, the classifications ofthe condition of each of the property characteristics may by output to acomputing device of a requester (424) via a dashboard interface screen.In some examples, in addition to providing the condition classificationfor the property characteristics, the dashboard interface screen mayinclude replacement costs for replacing the property characteristics,risk cost estimates due to damage from natural disasters, orconfirmation of whether or not repairs have been made to the propertycharacteristics.

If, instead, a classification match was not identified (420), in someimplementations, alternate two-dimensional or three-dimensional aerialimagery is accessed (410) and the method 400 restarts with overlayingthe replacement aerial imagery with the shape map (406). Alternatively,in some embodiments, even if a classification match is not identified,condition may be assessed based upon a default profile associated withthe property feature (not illustrated). For example, if the rooftopshape does not match one of the shapes trained into the system, thecondition of the rooftop may still be assessed based upon a generalcondition profile trained with a variety of rooftop shapes. In someexamples, the condition of the rooftop may be assessed by comparing thecurrent rooftop conditions to stored historic condition characteristicsfor the property from a previous condition assessment.

Although the method 400 is illustrated as a series of steps, in otherimplementations, more or fewer steps may be included. For example, insome implementations, terrestrial images are obtained and used toclassify the same and/or different property characteristics, asdescribed in relation to the operational flow 100 of FIG. 1.

Additionally, in other implementations, certain steps may be performedin a different order. For example, in some implementations, propertycharacteristics and condition characteristics may be classified (418,422) in parallel. Other modifications of the operational flow 100 arepossible.

Aspects of the present disclosure may be directed to computing systemsfor categorizing a repair condition of a property characteristic inorder to provide dynamic, real-time property condition assessments inresponse to requests received from users such as insurance carriers orreal estate investment brokers using aerial imagery. The implementationsdescribed herein improve upon conventional methodologies by applyingdeep learning analysis models to detected property characteristics ofthe obtained imagery in order to assess whether a condition of theproperty characteristics is degraded to a point of needing repair orreplacement and determine an amount of risk exposure of the property dueto the detected condition of the property characteristics. Theimplementations described herein improve processing efficiency of thesystem to reduce an amount of time it takes to perform the conditionassessments and automate a condition assessment process.

Next, a hardware description of the computing device, mobile computingdevice, or server according to exemplary embodiments is described withreference to FIG. 6. In FIG. 6, the computing device, mobile computingdevice, or server includes a CPU 600 which performs the processesdescribed above. The process data and instructions may be stored inmemory 602. These processes and instructions may also be stored on astorage medium disk 604 such as a hard drive (HDD) or portable storagemedium or may be stored remotely. Further, the claimed advancements arenot limited by the form of the computer-readable media on which theinstructions of the inventive process are stored. For example, theinstructions may be stored on CDs, DVDs, in FLASH memory, RAM, ROM,PROM, EPROM, EEPROM, hard disk or any other information processingdevice with which the computing device, mobile computing device, orserver communicates, such as a server or computer.

Further, a portion of the claimed advancements may be provided as autility application, background daemon, or component of an operatingsystem, or combination thereof, executing in conjunction with CPU 600and an operating system such as Microsoft Windows 6, UNIX, Solaris,LINUX, Apple MAC-OS and other systems known to those skilled in the art.

CPU 600 may be a Xenon or Core processor from Intel of America or anOpteron processor from AMD of America, or may be other processor typesthat would be recognized by one of ordinary skill in the art.Alternatively, the CPU 600 may be implemented on an FPGA, ASIC, PLD orusing discrete logic circuits, as one of ordinary skill in the art wouldrecognize. Further, CPU 600 may be implemented as multiple processorscooperatively working in parallel to perform the instructions of theinventive processes described above.

The computing device, mobile computing device, or server in FIG. 6 alsoincludes a network controller 606, such as an Intel Ethernet PRO networkinterface card from Intel Corporation of America, for interfacing withnetwork 6X. As can be appreciated, the network 628 can be a publicnetwork, such as the Internet, or a private network such as an LAN orWAN network, or any combination thereof and can also include PSTN orISDN sub-networks. The network 628 can also be wired, such as anEthernet network, or can be wireless such as a cellular networkincluding EDGE, 3G and 4G wireless cellular systems. The wirelessnetwork can also be Wi-Fi, Bluetooth, or any other wireless form ofcommunication that is known.

The computing device, mobile computing device, or server furtherincludes a display controller 608, such as a NVIDIA GeForce GTX orQuadro graphics adaptor from NVIDIA Corporation of America forinterfacing with display 610, such as a Hewlett Packard HPL2445w LCDmonitor. A general purpose I/O interface 612 interfaces with a keyboardand/or mouse 614 as well as a touch screen panel 616 on or separate fromdisplay 610. General purpose I/O interface also connects to a variety ofperipherals 618 including printers and scanners, such as an OfficeJet orDeskJet from Hewlett Packard.

A sound controller 620 is also provided in the computing device, mobilecomputing device, or server, such as Sound Blaster X-Fi Titanium fromCreative, to interface with speakers/microphone 622 thereby providingsounds and/or music.

The general purpose storage controller 624 connects the storage mediumdisk 604 with communication bus 626, which may be an ISA, EISA, VESA,PCI, or similar, for interconnecting all of the components of thecomputing device, mobile computing device, or server. A description ofthe general features and functionality of the display 610, keyboardand/or mouse 614, as well as the display controller 608, storagecontroller 624, network controller 606, sound controller 620, andgeneral purpose I/O interface 612 is omitted herein for brevity as thesefeatures are known.

One or more processors can be utilized to implement various functionsand/or algorithms described herein, unless explicitly stated otherwise.Additionally, any functions and/or algorithms described herein, unlessexplicitly stated otherwise, can be performed upon one or more virtualprocessors, for example on one or more physical computing systems suchas a computer farm or a cloud drive.

Reference has been made to flowchart illustrations and block diagrams ofmethods, systems and computer program products according toimplementations of this disclosure. Aspects thereof are implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in acomputer-readable medium that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablemedium produce an article of manufacture including instruction meanswhich implement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer implemented process such that theinstructions which execute on the computer or other programmableapparatus provide processes for implementing the functions/actsspecified in the flowchart and/or block diagram block or blocks.

Moreover, the present disclosure is not limited to the specific circuitelements described herein, nor is the present disclosure limited to thespecific sizing and classification of these elements. For example, theskilled artisan will appreciate that the circuitry described herein maybe adapted based on changes on battery sizing and chemistry, or based onthe requirements of the intended back-up load to be powered.

The functions and features described herein may also be executed byvarious distributed components of a system. For example, one or moreprocessors may execute these system functions, where the processors aredistributed across multiple components communicating in a network. Thedistributed components may include one or more client and servermachines, which may share processing, as shown on FIG. 8, in addition tovarious human interface and communication devices (e.g., displaymonitors, smart phones, tablets, personal digital assistants (PDAs)).The network may be a private network, such as a LAN or WAN, or may be apublic network, such as the Internet. Input to the system may bereceived via direct user input and received remotely either in real-timeor as a batch process. Additionally, some implementations may beperformed on modules or hardware not identical to those described.Accordingly, other implementations are within the scope that may beclaimed.

In some implementations, as illustrated in FIG. 7, the innovationsdescribed herein may interface with a cloud computing environment 730,such as Google Cloud Platform™ to perform at least portions of methodsor algorithms detailed above. The processes associated with the methodsdescribed herein can be executed on a computation processor, such as theGoogle Compute Engine by data center 734. The data center 734, forexample, can also include an application processor, such as the GoogleApp Engine, that can be used as the interface with the systems describedherein to receive data and output corresponding information. The cloudcomputing environment 730 may also include one or more databases 738 orother data storage, such as cloud storage and a query database. In someimplementations, the cloud storage database 738, such as the GoogleCloud Storage, may store processed and unprocessed data supplied bysystems described herein.

The systems described herein may communicate with the cloud computingenvironment 730 through a secure gateway 732. In some implementations,the secure gateway 732 includes a database querying interface, such asthe Google BigQuery platform.

The cloud computing environment 102 may include a provisioning tool 740for resource management. The provisioning tool 740 may be connected tothe computing devices of a data center 734 to facilitate the provisionof computing resources of the data center 734. The provisioning tool 740may receive a request for a computing resource via the secure gateway732 or a cloud controller 736. The provisioning tool 740 may facilitatea connection to a particular computing device of the data center 734.

A network 702 represents one or more networks, such as the Internet,connecting the cloud environment 730 to a number of client devices suchas, in some examples, a cellular telephone 710, a tablet computer 712, amobile computing device 714, and a desktop computing device 716. Thenetwork 702 can also communicate via wireless networks using a varietyof mobile network services 720 such as Wi-Fi, Bluetooth, cellularnetworks including EDGE, 3G and 4G wireless cellular systems, or anyother wireless form of communication that is known. In some embodiments,the network 702 is agnostic to local interfaces and networks associatedwith the client devices to allow for integration of the local interfacesand networks configured to perform the processes described herein.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the present disclosures. Indeed, the novel methods, apparatusesand systems described herein can be embodied in a variety of otherforms; furthermore, various omissions, substitutions and changes in theform of the methods, apparatuses and systems described herein can bemade without departing from the spirit of the present disclosures. Theaccompanying claims and their equivalents are intended to cover suchforms or modifications as would fall within the scope and spirit of thepresent disclosures.

What is claimed is:
 1. A method for automatically categorizing a repaircondition of a property characteristic, comprising: receiving, from auser at a remote computing device, a request for a property conditionclassification, wherein the property classification request includesidentification of a property and at least one property characteristic;obtaining, by processing circuitry of a computing system responsive toreceiving the request, an aerial image of a geographic region includingthe property; identifying, by the processing circuitry, a plurality offeatures of the aerial image corresponding to the propertycharacteristic; analyzing, by the processing circuitry, the plurality offeatures to determine a property characteristic classification;analyzing, by the processing circuitry, a region of the aerial imageincluding the property characteristic to determine a conditionclassification; determining, by the processing circuitry using theproperty characteristic classification and the condition classification,a risk estimate of damage to the property due to one or more disasters;and returning, to the user at the remote computing device via agraphical user interface responsive to receiving the request, acondition assessment of the property characteristic including thecondition classification and the risk estimate of damage to the propertydue to the one or more disasters.
 2. The method of claim 1, wherein theproperty characteristic classification is rooftop shape.
 3. The methodof claim 1, wherein analyzing the plurality of features to determine aproperty characteristic comprises applying a deep learning analysismodel to the plurality of features.
 4. The method of claim 1, whereinthe deep learning analysis model is NIN.
 5. The method of claim 1,wherein analyzing the region of the aerial image including the propertycharacteristic to determine the condition classification comprisesapplying a machine learning analysis model to a plurality of imagepixels within the region.
 6. The method of claim 5, wherein the machinelearning analysis model comprises a color histogram analysis model. 7.The method of claim 1, wherein determining the risk estimate comprisesapplying a disaster risk profile corresponding to a) a first disaster ofthe at least one disaster and b) the property characteristic.
 8. Asystem for automatically categorizing a repair condition of a propertycharacteristic, comprising: processing circuitry; and a non-transitorycomputer-readable medium having instructions stored thereon; wherein theinstructions, when executed by the processing circuitry, cause theprocessing circuitry to receive, from a user at a remote computingdevice, a property condition classification request, wherein theproperty classification request includes identification of a propertyand at least one property characteristic; obtain, from a remote datasource responsive to receiving the property classification request, anaerial image of a geographic region including the property; identify aplurality of features of the aerial image corresponding to the propertycharacteristic; analyze the plurality of features to determine aproperty characteristic classification; analyze a region of the aerialimage including the property characteristic to determine a conditionclassification; and determine, in real-time responsive to receiving theproperty classification request and using the property characteristicclassification and the condition classification, a replacement cost forreplacing the property characteristic.
 9. The system of claim 8, whereinthe instructions, when executed by the processing circuitry, cause theprocessing circuitry to, prior to identifying the plurality of features:obtain a shape map image including the property; overlay the aerialimage with the shape map image; and determine whether a boundary of theproperty as identified by the shape map matches a boundary of theproperty as illustrated in the aerial image.
 10. The system of claim 9,wherein the instructions, when executed by the processing circuitry,cause the processing circuitry to, upon determining that the shape mapdoes not match the boundary of the corresponding property, obtain analternate aerial image of the property.
 11. The system of claim 8,wherein the instructions, when executed by the processing circuitry,cause the processing circuitry to compare a current conditionclassification of the property characteristic to a stored historiccondition classification of the property characteristic.
 12. The systemof claim 11, wherein the instructions, when executed by the processingcircuitry, cause the processing circuitry to, in real-time responsive toreceiving the property condition classification request, determinewhether repairs have been made to the property based on the comparisonof the current condition classification to the historic conditionclassification of the property characteristic.
 13. The system of claim8, wherein the instructions, when executed by the processing circuitry,cause the processing circuitry to, prior to identifying the plurality offeatures, assess orthogonality of the aerial imagery.
 14. The system ofclaim 13, wherein the instructions, when executed by the processingcircuitry, cause the processing circuitry to, responsive to assessingthat the orthogonality of the aerial imagery does not correspond to atrue orthophoto form, cause correction of the aerial imagery to the trueorthophoto form.
 15. A non-transitory computer readable medium havinginstructions stored thereon, wherein the instructions, when executed byprocessing circuitry, cause the processing circuitry to: receive, from auser at a remote computing device, a property condition classificationrequest, wherein the property classification request includesidentification of a property and at least one property characteristic;obtain, from a remote data source responsive to receiving the propertyclassification request, an aerial image of a geographic region includingthe property; identify a respective plurality of features of the aerialimage corresponding to each property characteristic of the at least oneproperty characteristic; for each of the at least one propertycharacteristic, analyze the corresponding plurality of features todetermine a respective property characteristic classification, andanalyze a region of the aerial image including the respective propertycharacteristic to determine a respective condition classification; anddetermine, in real-time responsive to receiving the propertyclassification request and using the property characteristicclassification of each property characteristic and the conditionclassification of each property characteristic, at least one riskestimate representing risk of damage due to disaster.
 16. Thenon-transitory computer readable medium of claim 15, wherein theinstructions, when executed by the processing circuitry, cause theprocessing circuitry to, prior to obtaining the aerial image of thegeographic region, determine, based upon the at least one propertycharacteristic, a preferred image type corresponding to each propertycharacteristic of the at least one property characteristic.
 17. Thenon-transitory computer readable medium of claim 16, wherein: the atleast one property characteristic comprises two or more propertycharacteristics; and the instructions, when executed by the processingcircuitry, cause the processing circuitry to, responsive to determiningthe preferred image type corresponding to a first propertycharacteristic of the at least one property characteristic is aterrestrial image, obtain a terrestrial image of the geographic regionincluding the property.
 18. The non-transitory computer readable mediumof claim 15, wherein the instructions, when executed by the processingcircuitry, cause the processing circuitry to access one or more knownproperty characteristics, wherein determining the at least one riskestimate comprises determining the at least one risk estimate furtherbased on the one or more known property characteristics.
 19. Thenon-transitory computer readable medium of claim 18, wherein the one ormore known property characteristics comprise at least one of a propertyage, a property elevation, a property slope, a year built, a yearrenovated, and a building height.
 20. The non-transitory computerreadable medium of claim 15, wherein: receiving identification of theproperty and the at least one property characteristic comprisesreceiving, via a network from a remote computing device, theidentification of the property; and the instructions, when executed bythe processing circuitry, cause the processing circuitry to provide, viathe network to a graphical user interface at the remote computing devicein real-time responsive to receiving the at least one propertycharacteristic, the at least one risk estimate.